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PLANNING  SKILL  DEVELOPMFA'T 
FOR  SYSTEMS  ANALYSTS* 


To  motivate  and  retain  their  skilled  Systems 
Analysts,  DP  management  must  reassess  their 
skill  priorities,  skill  development  programs, 
and  personnel  evaluation  systems. 

"Grow  -your-  own  and  keep  them" 

Simply  stated,  the  demand  for  skilled  DP  personnel  is  greater  than  the 
available  supply;  competition  in  hiring  skilled  people  drives  up  salaries, 
and  has  left  most  DP  departments  without  enough  skilled  personnel.   Conse- 
quently, more  and  more  DP  departments  are  revitalizing  their  skill  develop- 
ment programs  and  are  reassessing  their  personnel  evaluation  procedures. 

Unfortunately,  our  research  results  reveal  a  hidden  crisis.   Skill 
development  programs  omit  major,  relevant  topics  and  do  not  reward  systems 
analysts  for  developing  top  priority  skills.   Personnel  evaluation  systems 
are  virtually  driving  systems  analysts  out  of  some  firms  with  obsolete  stan- 
dards, and  are  not  rewarding  systems  analysts  for  the  skills  the  head  of  the 
DP  department  rates  as  most  important. 

Success  requires  a  practical  planning  technique  for  the  diagnosis  and 
improvement  of  current  problems.   DP  management  must  identify  the  relevant 
skills  for  an  SA,  modify  the  evaluation  system  to  reward  these  skills,  com- 
municate these  priorities  effectively,  and  assure  at  least  the  existence  of 
a  skill  development  program  in  top  priority  skills. 

These  necessities  for  success  may  seem  obvious,  but,  according  to  our 
research,  they  have  not  been  practiced  in  even  better  companies.   In  order 
to  examine  these  issues  we,  at  the  Center  for  Information  Systems  Research, 
surveyed  systems  analysts  and  heads  of  DP  departments  about  skill  priorities 
and  evaluation  standards  for  systems  analysts. 


*  We  wish  to  thank  the  anonymous  managers  and  companies  who  participated  in 
this  research,  Christine  Bullen  for  her  management  of  the  data  gathering 
process,  and  the  Center  for  Information  Systems  Research  for  partial  funding. 

This  is  part  of  a  larger  research  project.   See  "User  Needs  Survey:   Preli- 
minary Results",  Robert  M.  Alloway,  et_  al_,  MIT,  Sloan  Working  Paper  1096-79, 
December  1979. 
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Prioritization  of  Skills 

Our  research  began  with  identification  of  a  list  of  relevant  skills 
for  a  systems  analyst  (SA) .   We  designed  the  list  to  be  short  but  represen- 
tative of  the  broad  range  of  relevant  skills.   If  a  particular  company  wanted 
to  develop  a  relevant  list  of  skills  and  perform  an  internal  diagnostic  com- 
parable to  this  research  project,  it  could  easily  be  done  by  customizing  our 
procedures. 

The  sixteen  skills  used  in  our  survey  and  throughout  this  article  are 
listed  in  Table  1.   They  were  randomly  sequenced  on  a  questionnaire,  and 
respondents  rated  each  on  the  seven-point  scale  shown,  where  1  represents  a 
completely  irrelevant  skill,  3  a  necessary  prerequisite,  5  very  important, 
and  7  the  single  most  important  skill. 

The  Chief  Information  Officers  (CIO)  in  companies  V,  W,  X,  Y,  and  Z 

2 
rated  the  priority  of  each  skill.   The  order  in  which  the  sixteen  skills 

are  numbered  in  Table  1  is  the  average  value  of  each  skill  on  the  seven 
point  scale  of  the  CIOs'  priority  ratings. 

We  do  not  contend  the  CIOs  surveyed  are  right  in  their  priority  assign- 
ments, and  the  proper  priorities  of  the  skills  do,  of  course,  vary  by  com- 
pany. However,  our  data  does  show  a  basic  agreement  among  these  five  CIOs 

3 
on  their  priority  assignments  of  systems  analyst  skills. 

The  bottom  six  priorities  (rated  not  as  irrelevant,  but  as  necessary 
prerequisites)  are  dominated  by  the  traditional  technical  skills,  whereas 
the  top  six  all  relate  to  a  strong  user  orientation.   The  CIOs'  emphasis 
is  on  managerial  and  organizational  issues  at  the  front  end  of  the  systems 
development  life  cycle. 

It  seems  likely  that  the  current  priorities  of  these  CIOs  have  evolved 
only  recently.  However,  changes  in  priorities  should  be  effectively  commu- 
nicated to  the  analysts  involved  and  reflected  in  modified  evaluation  stan- 
dards. 
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CIO  is  our  term  for  the  head  of  the  DP  department;  typically,  this  is  the 

Director  or  Vice  President  of  DP,  MIS  or  Information  Services. 
3 

The  Pearson  R  correlation  coefficients  of  each  CIO's  ratings  to  average 

CIO  ratings  are  .68,  .69,  .59,  .83,  and  .72. 


SA,  CIO  Skill  Priorities  Compared 

To  investigate  the  extent  to  which  the  CIO's  priorities  have  been 
reflected  in  his  department,  we  compared  the  skill  priorities  of  the  CIO 
in  company  Z  with  the  priorities  of  one  of  his  SAs  for  each  skill  in  Table  1. 
These  two  people  form  one  of  the  pairs  of  CIO-SA  studied.   Their  level  of 
agreement  on  skill  priorities  is  shown  in  Figure  1. 

If  their  agreement  had  been  perfect,  both  bars  would  be  the  same  height 
for  all  skills,  one  to  sixteen.   It  is  easy  to  see  the  extent  of  disagreement 
for  this  pair  and  consider  the  implications  for  productivity  and  successful 
software  development. 

For  example,  the  CIO  rates  implementation  planning,  education,  moti- 
vation, and  training  of  users  (skill  #9)  as  very  important,  whereas  this 
SA  considers  it  irrelevant.   Conversely,  this  SA  rates  technical  skills 
(skill  #16)  as  very  important,  while  his  CIO  considers  it  only  a  necessary 
prerequisite.   To  the  extent  that  a  CIO-SA  pair  disagree  on  skill  priorities, 
considerable  efforts  may  be  misal located ,  allocated  at  cross  purposes,  or 
important  issues  may  be  unintentionally  ignored.   This  is  because  an  SA'a 
skill  priorities  indicate  what  aspects  of  the  systems  development  process 
he/she  would  emphasize  --  in  this  example,  technical  considerations  over 
implementation  planning. 

For  each  of  the  CIO-SA  pairs  studied,  a  Figure  1  can  be  prepared.   To 
compare  all  CIO-SA  pairs  studied,  we  needed  a  composite  index  for  level  of 
agreement  on  the  skills  one  to  sixteen.   U'e  used  a  statistic  called  correla- 
tion coefficient  which  ranges  in  value  from  +1  for  perfect  agreement  to 

-1  for  perfect  disagreement.   Figure  1  has  a  correlation  coefficient  of 

4 
.22  but  is  not  statistically  significant. 


For  the  16  skills  and  variance  in  ratings  assigned  by  SAs  and  CIOs,  a 
Pearson  R  correlation  coefficient  of  .50  or  greater  is  statistically  sig- 
nificant at  the  .05  level.   Statistical  significance  indicates  that  there 
is  a  relationship  between  priorities  (S-o  chance  there  is  no^  relationship)  , 
however,  it  does  not  indicate  \.he  strength  of  agreement.   A  correlation 
coefficient  of  .80  is  strong  agreement  and  is  statistically  significant. 
A  correlation  coefficient  of  .50  is  statistically  significant  but  indi- 
cates only  weak  agreement.   Any  lower  correlation  coefficient,  for  example 
.22,  indicates  no  agreement,  i.e.,  only  a  random  relationship. 


Company  averages  for  the  level  of  skill  priority  agreement  by  CIO-SA 
pair  were:   .11,  .11,  -.19,  .in,  and  .22.   The  highest  company  average 
correlation  coefficient  is  .22,  coincidentally  the  sane  as  the  pair  in 
Figure  1,  and  simply  managerial ly  unacceptable. 

To  look  at  all  the  CIO-SA  pairs  studied  we  prepared  the  histogram  in 
Figure  2.   The  correlation  coefficient  for  skill  priority  agreement  is  the 
horizontal  axis  and  the  height  of  the  bars  indicates  the  relative  number  of 
pairs  with  that  level  of  agreement. 

The  particular  pair  in  Figure  1,  with  a  correlation  coefficient  of 
.22,  is  novNT  seen  to  be  t\'pical  and  above  average  for  all  pairs.   The  pairs 
with  reasonable  agreement,  the  9%  with  positive  and  statistically  significant 
correlations,  are  few  and  far  between.   For  the  other  91''<,  of  the  CIO-SA  pairs, 
their  level  of  agreement  on  skill  priorities  is  random.   Random  --  think 
about  that  for  a  minute. 

Clearly,  more  effective  communication  and  discussion  of  skill  priorities 
for  SAs  is  necessary.   Productivity  includes  doing  the  right  thing  as  well 
as  doing  it  efficiently.  Motivation  relates  to  doing  something  person- 
ally considered  important.   It  is  hard  enough  to  lead  when  everyone  agrees 
on  the  direction.   With  our  survey  results,  it  is  easy  to  see  why  both  leaders 
and  followers  in  DP  departments  feel  frustrated. 

Perceived  Evaluation  Standards 


Each  Systems  Analyst  also  rated  the  16  skills  for  perceived  evaluation 
standards  for  positive  feedback,  raises,  and  promotions.   IVe  have  compared 
these  ratings  with  CIO  skill  priorities  to  see  if  important  skills  are 
being  rewarded. 

The  perceived  evaluation  standards  are  reality  for  a  systems  analyst. 
It  is  possible  that  an  SA  may  have  a  mis-perception  of  the  actual  evaluation 
standards  in  his/her  own  DP  department.   More  likely,  however,  DP  management 
has  been  too  busy  fulfilling  users  needs  to  revise  its  own   obsolete  evalua- 
tion standards.   Even  with  revised  standards,  the  evaluation  system  must  be 
well  implemented  to  send  clear,  consistent  signals  to  SAs.   It  is  the  per- 
ception of  what  is  rewarded  that  influences  behavior. 


Selecting  the  same  CIO-SA  pair  from  Company  Z,  we  have  displayed  the 
level  of  agreement  between  CIO  skill  priorities  and  SA  perceived  evaluation 
standards  in  Figure  5.   If  agreement  were  perfect,  the  bars  would  be  the 
same  height  for  all  sixteen  skills.   The  disagreements  evident  in  Figure  3 
indicate  the  CIO  is  being  actively  subverted  by  his  own  department's  evalua- 
tion system.   Moreover,  this  SA  is  being  encouraged  to  develop  skills  the 
CIO  considers  virtually  irrelevant. 

The  CIO  considers  implementation  planning  (skill  #9)  to  be  very  important. 
Unfortunately,  this  skill  is  not  being  effectively  rewarded.   The  SA  perceives 
skills  in  implementation  planning  to  be  virtually  ignored  by  the  evaluation 
system  and  will  behave  accordingly. 

Conversely,  the  SA  perceives  skills  5  and  14  to  be  equally  important. 
Unfortunately,  the  CIO  considers  skill  14  (estimating  and  rigid  adherence 
to  project  costs  and  schedules)  virtually  irrelevant  and  skill  3  (organi- 
zational design  and  assessing  system  impacts  on  user  departments)  very 
important  and  will  behave  accordingly. 

The  result  is  the  SA  finds  the  evaluation  system,  and  his  CIO,  to  be 
arbitrary  and  capricious,  while  the  CIO  finds  the  evaluation  system,  and 
his  SA,  to  be  ineffective  in  improving  or  changing  skills  or  behavior. 

The  level  of  agreement  in  Figure  5  expressed  as  a  correlation  coefficient 
is  .08.  The  average  correlation  coefficient  for  all  CIO-SA  pairs  in  each 
company  are:   -.02,  ,20,  -.14,  .13,  and  .14.   These  results  are  unacceptable 
for  all  five  companies.   It  is  clearly  necessary  for  these  CIOs  to  devote 
some  portion  of  their  busy  schedules  to  a  revision  of  the  evaluation  system, 
its " implementation ,  and  effective  coiranunication. 

The  skill  priorities  of  the  CIOs  have  precious  little  affect  on  what 
gets  rewarded  in  their  own  departments.   Figure  4  makes  this  point  drama- 
tically by  summarizing  the  level  of  agreement  for  all  CIO-SA  pairs  studied. 
Only  15%  of  these  pairs  have  a  statistically  significant  correlation  between 
CIO  skill  priorities  and  SA  perceived  evaluation  standards.   The  rest,  87%, 
are  random,  and  more  than  50%  of  all  the  pairs  are  worse  than  the  pair  de- 
tailed in  Figure  5. 


We  could  have  selected  CIO  skill  priorities  with  a  random  number  genera- 
tor and  had  an  equal  chance  of  comparable  agreements  for  87°6  of  the  CIO-SA 
pairs.  Klien  perceived  evaluation  standards  do  not  agree  with  intended 
priorities,  the  effect  is  to  both  stimulate  and  reinforce  undesired  behavior, 
generating  the  cross  purposes  and  frustrations  discussed  for  Figure  3  on  a 
massive  scale. 

The  Evaluation  System  is  Key 

When  an  individual's  perceived  evaluation  standards  do  not  agree  with 
his/her  own  skill  priorities,  considerable  frustration  is  generated.   This 
frustration  is  manifested  in  lower  job  satisfaction,  motivation  and  producti- 
vity, and  a  higher  propensity  to  quit. 

We  don't  really  know  what  level  of  agreement  between  an  SA's  skill 
priorities  and  perceived  evaluation  standards  is  adequate  to  prevent 
frustration.   For  example.  Figure  5  displays  the  effect  of  perceived  evalua- 
tion standards  on  skill  priorities  for  our  favorite  SA  from  Company  Z.   There 
are  regular  but  reasonable  differences  on  the  16  skills.   Only  skills  7  and 
13  are  out  of  balance,  and  this  doesn't  appear  to  be  sufficient  to  cause 
a  real  problem. 

Figure  5  has  a  very  high  and  statistically  significant  correlation 
coefficient  of  .87.   The  company  averages  are  generally  good:   .52,  .60, 
.29,  .65,  and  .67.   Even  without  knowing  the  exact  threshold  level  of  dis- 
agreement for  frustration,  it  is  easy  to  see  why  some  companies  experience 
higher  SA  turnover  than  others.   Company  X,  with  a  .29  average,  is  definitely 
driving  SAs  out  of  their  DP  department.   IvTien  SAs  are  hearing  the  abandon 
ship  alarm,  it  is  not  the  time  to  rearrange  the  deckchairs  on  the  Titanic. 

There  is  a  very  strong  relationship  between  an  SA's  perception  of 
evaluation  standards  and  his/her  own  skill  priorities.   Figure  6  summarizes 
the  correlation  coefficients  for  all  SAs  studied.   Sixty-one  percent  of  the 
relationships  are  statistically  significant. 


These  are  encouraging  results.   There  is  no  doubt  that  evaluation/reward 
systems  work.   Revision  of  the  evaluation  standards,  their  implementation 
and  effective  communication  are  managerially  feasible  and  will  affect  SA's 
perceptions,  skill  priorities,  motivation,  productivity,  skill  development, 
and  turnover.   However,  there  are  three  aspects  of  this  strong  relationship 
between  perceived  evaluation  standards  and  skill  priorities  which  must  be 
considered  very  carefully  in  planning. 

First,  some  people  leave  an  organization  rather  than  modify  their 
behavior  in  accordance  to  evaluation  standards  with  which  they  strongly 
disagree.   Consequently,  we  do  not  know  what  proportion  of  this  strong 
relationship  is  due  to  self-selection.  More  importantly,  we  do  not  know 
what  proportion  of  the  current  SAs  would  choose  to  leave  if  they  perceived 
a  major  change  in  evaluation  standards. 

Second,  as  Figure  6  demonstrates,  evaluation  standards  are  quite 
influential.   The  evaluation/reward  system  not  only  influences  overt  behavior 
but,  over  time,  influences  deep-seated  personal  opinions  like  skill  priorities. 
Conversely,  changing  personal  opinions  requires  equivalent  influehce  over  time. 
A  quick,  light  fix  would  be  ineffectual. 

Third,  this  strong  relationship  is  not  necessarily  good.   In  fact,  for 
these  five  companies  it  is  making  matters  worse.   We  have  already  demonstrated 
that  the  perceived  evaluation  standards  are  wrong  --  they  disagree  with  the 
skill  priorities  defined  as  desirable  by  the  CIO.   Remember,  Figure  3  revealed 
a  basically  random  relationship  between  SA  perceived  evaluation  standards  and 
CIO  skill  priorities. 

The  evaluation/reward  system  does  work.   But,  being  obsolete,  it  is 
driving  SAs  away  from  their  CIO's  evolving  priorities.   It  is  sending  the 
wrong  signals  throughout  the  department,  emasculating  the  skill  development 
program,  rewarding  the  wrong  skills,  mis-directing  emphasis  in  systems 
development,  and  driving  the  wrong  SAs  out  of  the  DP  department.   In  this 
situation,  the  longer  the  delay  in  modifying  the  evaluation/reward  system, 
the  worse  the  problem. 


Conclusions 

These  results  are  so  strong  and  nearly  uniform  across  the  five  companies 
that  they  can  probably  be  generalized  to  a  larger  number  of  DP  departments. 
Managerial  actions  to  improve  this  situation  can  and  should  be  taken. 

The  CIO  and  his  DP  management  team  must  conscientiously  and  explicitly 
identify  and  prioritize  skills  for  SAs.   To  the  extent  that  the  CIO's  priori- 
ties are  uncertain  or  evolving  over  time,  a  continuous  and  effective  communi- 
cation and  discussion  process  must  be  established. 

The  current  situation  in  DP  must  be  diagnosed.   A  customized  version  of 
the  straightforward  methods  used  in  our  research  could  be  used  for  an  internal 
diagnostic  and  discussion  vehicle. 

The  longer  the  delay,  the  more  obsolete  evaluation  standards  influence 
SAs  away  from  the  CIO's  priorities.   Moreover,  the  duplicity  of  the  situation, 
the  CIO's  espoused  priorities  versus  perceived  evaluation  standards, 
understandably  frustrates  SAs.  The  result  is  misallocated  and  cross-purpose 
efforts,  lower  productivity  and  motivation,  and  higher  turnover. 

The  evaluation  standards  must  be  revised  to  agree  with  the  desired 
priorities.   This  entails  revision  of  the  evaluation/reward  system,  effective 
communication  of  the  new  evaluation  standards,  and  clear,  consistent  signals 
from  its  implementation.   During  this  process,  SAs  will  experience  pressure 
for  change.  Your  plan  should  anticipate  disbelief  with  the  new  priorities 
and  evaluation  standards  until  actual  practice  has  demonstrated  real  imple- 
mentation. 

If  the  current  skill  development  program  is  both  irrelevant  to  organi- 
zational rewards  and  SAs'  skill  priorities,  its  impact  on  SAs'  skills, 
irrespective  of  level  of  expenditure,  is  quite  predictable. 

The  CIO's  skill  priorities  should  drive  the  emphasis  and  offerings  in 
the  skill  development  program.   Most  skill  development  programs  concentrate 
on  the  traditional  technical  skills  found  in  the  bottom  six  priorities  in 
Table  1.   These  are  indeed  "necessary  prerequisite"  skills.  However,  the 
skill  development  program  should  not  stop  there;  rather,  a  concerted  effort 
to  identify  program  elements  (courses  and  task  assignments)  for  the  develop- 
ment of  higher  priority  skills  is  required.   The  revised  evaluation  system 
will  stimulate  SAs'  interest  in  top  priority  skills  and  their  development. 


The  shortage  of  skilled  DP  personnel  is  definitely  a  long-term  problem. 
It  is  clearly  desirable  for  the  CIO,  manager  of  systems  development,  and 
SAs  to  identify  relevant  skills  and  agree  on  their  priorities  for  SAs.   The 
skill  development  program  and  evaluation/reward  system  should  develop  and 
reinforce  these  skills  effectively.   Unfortunately,  our  research  results 
demonstrate  that  this  is  not  t>'pical  practice.  A  company  specific  diagnostic 
could  be  performed  in  any  DP  department  by  customizing  the  research  approach 
we  used.   However,  for  most  DP  departments  the  recommendations  we  have  dis- 
cussed could  be  directly  applied  --  yielding  significant  benefits  by  closing 
the  gap  between  typical  and  desired  practice. 


Table  1 

.   Systems  Analysts'  Skills* 

completely         a  necessary  very  single  most 

irrelevant         prerequisite  important        important  skill 


Average  CIO 
Priority  Rating 

6.0   1.  strong  user  orientation,  ability  to  work  with  users,  and  delivering  systems  users 
really  like 

5.4  2.  ability  to  work  with  ill-defined  objectives  and  resolve  conflict  productively 

5.0  3.  skills  in  organizational  design  and  assessing  system  impacts  on  user  departments 

4.6  4.  behavioral  sensitivity  to  impacts  of  systems  on  individuals 

4.4  5.  project  management  skills  (planning  and  control) 

4.4  6.  broad  view  of  company  goals  and  operations,  senior  management  orientation 

4.0  7.  in-depth  knowledge  of  user  departments'  operations 

4.0  8.  dedication,  hard  work,  and  hustle 

4.0  9.  implementation  planning,  education,  motivation,  and  training  of  users 

4.0  10.  cost  consciousness,  hardware  and  operational  efficiency 

3.6  11.  expertise  in  system  desicn  of  monitor  and  exception  systems 

3.4  12.  leadership  ability,  administrative  experience,  sensitivity  to  political  issues 

3.4  13.  ability  to  work  intimately  with  senior  user  managers 

3.0  14.  estimating  and  rigid  adherance  to  project  costs  and  schedules 

2.8  15.  expertise  in  system  desion  of  inquiry  and  analysis  systems 

2.6  16,  technical  skills  (programming,  database  design,  telecommunication,  etc.) 


*CIOs  rated  the  priority  of  each  skill  for  SAs  using  the  seven  point  scale  above. 
The  list  has  been  re-sequenced  and  re-numbcrcd  by  average  priority  of  CIOs. 
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FIGURE    2 
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SKILL  PRIORITY  AGREEfiENT 

CIO  Skill  Priority  vs  SA  Skill  Priority 

summary  of 
All  CIO-SA  Pairs  in  all  Companies 


number  of 

cio-sa 

pairs 


-.3 


I/O 


-i         i 1 — 

-.2       -.1       0      .1 


J  ,        ' 


-I — ■ — r 


PAIR  DETAILED   IN  FIGURE  1 


Wo  SIGNIFICATfT 
|- >► 


1 r 


"1 r 


.2        .3       .4       .5      .6       .7 
LEVEL  OF  AGREEMENT   (CORRECTION  COEFFICIENT) 


.8 


1 
.9 


FIRM  AX'ERAGES 


V         W       X        Y         Z 

.II       .U    -.19      .10       .22 


CO 
Q 

< 


oo 


1— 

•—I 

rvj 

'—^ 

\- 

LU 

< 

>- 

:a:_ 

r) 

2 

LU 

_j 

< 

LU 

< 

CL 

Dl 

> 

s: 

ti3 

LU 

o 

<c 

Q 

c_> 

oo 

LU 

s: 

>o 

en 

> 

o 

cc: 

q: 

LU 

=a; 

uu 

q:    u. 

a: 

(=1 

o 

O 

D 

p=^ 

d: 

u.    cc 

<s> 

•=c 

UJ 

•— 1 

1— 

Cl_ 

< 

u. 

oo 

<i: 

Q_ 

:=- 

oo 

<C 

o 

OO 

•—1 

ir> 

1 

F— 

> 

o 

<i: 

zz> 

>• 

C-J) 

1 

\- 

<r 

•— » 

LU 

> 

cc 

■21 

LU 

o 

d: 
n 

-I 

o 

oo 
o 

CO 
CD 


LU 

a! 
n. 

LU 

o 


LU 

a: 
g 


1& 


[1 


13 


13 


^^ 


l\\\\\\\\\\\\\\\V^ 


\\\\\\\\N 


kv\\\m\\\\\\K\\\\^ 


P 


IWVWWWWWWWXT 


t^\\\\\\\\\\\\\\\\\\\\\\\\V\V\\\\\\\VVY^ 


UD 


Ln 


N^ 


— I— 

CSJ 


i£> 


LO 


CO 


I—    CL 

CO 

LU 

O    Q 


cr.    C/3 
O 


CO 


VD 


LD 


CO 


CM 


LU 


LU 
LU 
CD 


CO 
LU 


d 

to 


CO 

g 

< 


oo 

o 


< 


LU 
U 
CC 
lU 

Q_ 


D 


o 


q: 


oo 


en 


LU  o 
_J  D- 


w 


g 


>-  QJ 

tr  I— 
<  >^ 

CO  00 
CO  — 

LU  r) 
CJ  a 

LU  LU 

2  a: 
uu 
en 
a. 


LU  < 

-J  _l 

CL  lU 

O  OH 


FIGURE  ^ 

EVALUATION  STANDARDS  AGREEMENT 

CIO  Skill  Priorities  vs  SA  Perceived  Evaluation  Standards 

summary  of 
All  CIO-SA  Pairs  in  all  Companies 
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FIGURE   5 

SA  SELF  AGREEMENT 

SA  Perceived  Evaluation  vs  SA  Skill  Priorities 

summary  of 
All  SAs  in  all  Companies 
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